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COMMENT ON 'A DECOMPOSITION OF SCHUR FUNCTIONS AND AN 
ANALOGUE OF THE ROBINSON-SCHENSTED-KNUTH ALGORITHM' 

S. MASON 



The purpose of this comment is to clarify the connections between Demazure characters and the objects 
studied in this work. The nonsymmetric Macdonald polynomials introduced by Macdonald [7] and studied 
by Chcrcdnick [T] are denoted by Ea{X;q,t), where a is a weak composition and X = {xi,X2, ■ ■ ■)■ The 
Demazure characters introduced by Demazure in [5] and studied by Ion 0, Joseph [1], and Sanderson |10| 
are the specializations Ea{X; 0, 0). 

Marshall works with a variation of the above nonsymmetric polynomials obtained by reversing 
the indexing composition, reversing the variables, and replacing q and t by q~^ and t~^ respectively. 
These nonsymmetric polynomials, denoted Ea{X;q,t), can therefore be written as Ea{xi,X2T ■ -^qyt) = 
Erovcisc{a)i- ■ ■ j 3^2, s^i; q^^,t^^). It is these polynomials that we specialize to obtain the polynomials explored 
in this paper. In fact, the specializations of the Ea{X;q,t) to q = t = are equivalent to the second 
family of Demazure characters, often called "standard bases" or "Demazure atoms" , introduced by Lascoux 
and Schiitzenberger in [5] and studied by Lascoux in [5]. Please see [5] for a combinatorial proof of this 
equivalence. 

We provide the following short table for the partition A = (2, 1,0) to illustrate the distinction between 
Ea{X;0,0) and Ea,{X;0,0). 



Composition a 


Ea{X;Q,0) 


£;„(X;0,0) 


(2,1,0) 


X(X2 


X{X2 


(2,0,1) 


x(x2 + x'fxs 


2:^2:3 


(1,2,0) 


x(x2 + Xix'i 


2:12:^ 


(1,0,2) 


x{x2 + XiXg + x'ix'i + 2:12:20:3 + 2:12:3 


2:1X22:3 +2:12:^ 


(0,2,1) 


2:12:2 + 2-12:2 + 2:12:3 + 2:12:22:3 + 2:22:3 


a;ia:22:3 + 2:32:3 


(0,1,2) 


2:12:2 + 2:i2:| + 2:^2:3 + 22:12:22:3 + 2:22:3 + 2:12:3 + 2:22:3 


2:22:3 



X 



References 

[1] Chcrcdnik, I,. Nonsymmetric Macdonald polynomials. Math. Res. Notices, 10 (1995), pp. 483—515. 

[2] Demazure, M., Desingularisation des varietes de Schubert, Ann. E. N. S., 6 (1974), 163-172. 

[3] Ion, Bogdan, Nonsymmetric Macdonald polynomials and Demazure characters, Duke Mathematical Journal, 116 (2003), 

299-318 
[4] Joseph, A., On the Demazure character formula, Ann. Sci. Ecole Norm. Sup. (4), 18:3 (1985), 389-419. 
[5] Lascoux, A., and Schiitzenberger, M.-P., Keys and Standard Bases, Invariant Theory and Tableaux, IMA Volumes in Math 

and its Applications (D. Stanton, ED.), Southend on Sea, UK, 19 (1990), 125-144. 
[6] A. Lascoux, Double Crystal graphs. Studies in Memory of Issai Schur, Progress In Math. 210, Birkhaiiser (2003) 95-114. 
[7] Macdonald, I. C, Affine Hecke algebras and orthogonal polynomials, Astcrisquc 237 (1996), pp. 189-207, Scminaire Bour- 

baki 1994/95, Exp. no. 797. 
[8] Dan Marshall, Symmetric and nonsymmetric Macdonald polynomials. On combinatorics and statistical mechanics, Ann. 

Comb. 3 (1999), no. 2-4, 385-415. 
[9] Mason, S., An explicit construction of type A Demazure Atoms, to appear in J. Algebraic Combinatorics. 
[10] Sanderson, Y., On the Connection between Macdonald polynomials and Demazure characters, J. Algebraic Combin. 11 

(2000), no.3, 269-275. 

1 



S. MASON 



Department of Mathematics, Davidson College 
E-mail address: saiiiason@davldson.edu 
URL: http : //www . davldson . edu/math/mason 



(N 



A DECOMPOSITION OF SCHUR FUNCTIONS AND AN ANALOGUE 

- - -. OF THE ROBINSON-SCHENSTED-KNUTH ALGORITHM 

0\ 
O 
S ■ S. MASON 

Qh' Abstract. We exhibit a weight-preserving bijection between semi-standard Young 

■^T I tableaux and semi-skyhne augmented filhngs to provide a combinatorial proof that the 

Schur functions decompose into nonsymmetric functions indexed by compositions. The 
insertion procedure involved in the proof leads to an analogue of the Robinson-Schensted- 
Knuth Algorithm for semi-skyline augmented fillings. This procedure commutes with the 
s^ i Robinson-Schensted-Knuth Algorithm, and therefore retains many of its properties. 

U 

■3 
^: 

1. Introduction 

K*- ! Given a partition A = (Ai, A2, . . ■), the Schur function sx = sx{x) in the variables x = 

15^ I (xi, X2, . . .) is the formal power series sx = XIt-^^' where the sum is over all semi-standard 

Tj- ■ Young tableaux T of shape A. A semi-standard Young tableau is a diagram consisting of rows 

^ . of squares such that the z*^ row contains Aj squares, called cells. This diagram, called the 

\Q ! Young (or Ferrers) diagram, is drawn in the first quadrant, French style, as in [T]. Each of 

O ' these cells is then assigned a positive integer entry so that the entries are weakly increasing 



^ 



X 



along rows and strictly increasing along columns, where the rows are read from left to right 
c^ . and the columns are read from bottom to top. The values assigned to the cells of A collectively 

2 I form the multiset {1"S 2"^, . . . , n""}, for some n, where Oj is the number of times i appears 

in T. The weight of the semi-standard Young tableau (SSYT) T is given by x'^ = YYi=i ^T ■ 
The sum of the weights of all SSYT of shape A is the Schur function sx- See [8] for a more 
detailed discussion of symmetric functions and Schur functions in particular. 
5^1 The Macdonald polynomials Hx{x]q,t) are a class of functions symmetric in a; = 

(xi,a;2, . . .) with coefficients in Q(g, t). Macdonald introduced them in [5] and conjectured 
that their expansion in terms of Schur polynomials should have positive coefficients. A com- 
binatorial formula for the Macdonald polynomials is proved by Haglund, Haiman, and Loehr 
in [2] . A corresponding combinatorial formula for the integral form Macdonald polynomials, 
Jx, is obtained from the plethystic substitution which transforms the Hx polynomials into 
integral form. 
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2 S. MASON 

Building on this work, Haglund, Haiman, and Loehr [3] derive a combinatorial formula 
for nonsymmetric Macdonald polynomials, which gives a new decomposition of J\ into non- 
symmetric components. Letting g = t = oo in this decomposition and using the fact that 
Ja(x; q, q) = Sx{x) produces the identity 

(1.1) sa(x) = J]^^(x;0,0), 

summed over all rearrangements 7 of the partition A, denoted 7 ~ A. (A rearrangement of a 
partition A is an ordered sequence of non-negative integers (a weak composition), 7, whose 
parts, when rearranged into weakly decreasing order, form the partition A.) Here E^{x; 0, 0) 
is the specialization of the nonsymmetric Macdonald polynomial studied by Marshall [6]. (We 
use the form E instead of E and replace q and t with q~^ and t~^ because the associated 
combinatorics is more elegant in this setting.) The polynomial E^{x]0,0) can be described 
combinatorially through objects called semi-skyline augmented fillings constructed using the 
statistics from [5]. 

We provide a combinatorial proof of this decomposition of the Schur functions in Section [3] 
by constructing a weight-preserving bijection \1/ between semi-standard Young tableaux and 
semi-skyline augmented fillings. We build a semi-skyline augmented filling through an inser- 
tion procedure similar to Schensted insertion. This procedure is the fundamental operation 
in an analogue of the Robinson-Schensted-Knuth algorithm. 

Theorem 1.1. There exists a bijection between N— matrices with finite support and pairs 
{F,G) of semi- skyline augmented fillings of compositions which rearrange the same partition. 

This bijection is described in Section HI We prove that it commutes with the Robinson- 
Schensted-Knuth (RSK) algorithm and retains the symmetry of the RSK algorithm plus 
many of its other properties. We also describe the standardization of a semi-skyline aug- 
mented filling (SSAF) and prove that the RSK analogue commutes with standardization. 

2. Combinatorial description of the functions E^{x;0,0) 

The functions E^{x]0,0) are obtained from the integral form nonsymmetric Macdonald 
polynomials E^{x\q,t) by letting q and t approach zero. Haglund, Haiman, and Loehr 
provide a combinatorial formula for nonsymmetric Macdonald polynomials [3] which can be 
specialized to obtain a combinatorial formula for nonsymmetric Schur functions. Several 
definitions are necessary in order to describe this formula. 

2.1. Statistics on fillings. Let 7 = (71,72, . . .) be a weak composition of n into m parts. 
The column diagram of ^ is the figure dg'{'y) consisting of n cells arranged into columns, as 
in [3]. The i*'^ column contains 7, cells, and the number of cells in a column is called the 
height of that column. A cell a in a column diagram is denoted a = {i,j), where i is the 
column index and j is the row index. 
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The augmented diagram of 7, defined by dg{'-f) = dg'{'-f) U {(^,0) : I < i < m} is the 
column diagram with m extra cells adjoined in row 0. In this paper the adjoined row, called 
the basement, always contains the numbers 1 through m in strictly increasing order. (We 
italicize the basement entries to avoid confusion.) 

The augmented diagram for 7 = (0, 2, 0, 3, 1, 2, 0, 0, 1) is depicted below. 

dg(7) = 





























1 


2 


3 


4 


5 


6 


7 8 


9 



An augmented filling, a, of an augmented diagram dgi^'j) is a function a : dgi^^y) -^ Z+, 
which can be pictured as an assignment of positive integer entries to the cells of 7. Let a{i) 
denote the entry in the z*'^ cell of the augmented diagram encountered when dg{'-f) is read 
across rows from left to right, beginning at the highest row and working down to the bottom 
row. This ordering of the cells is called the reading order. A cell a = {i,j) precedes a cell 
b = {i',j') in the reading order if either j' < j or j' = j and i' > i. The reading word readia) 
is obtained by recording the non-basement entries in reading order. The content of a filling 
a is the multiset of non-basement entries which appear in the filling. (See Figure [2m ) 

A pair of cells a and b are called attacking if either a and b are in the same row or a and 
b are in adjacent rows, with the entry in the higher row strictly to the right of the entry 
in the lower row. (That is, a = («i,Ji) and b = (^2,^2) are attacking if either ji = J2, or 
J2 — Ji = 1 and ii < 12, or ji — j 2 = 1 and 12 < ii-) A non-attacking filling is a filling such 
that a{a) 7^ a{b) for every pair of attacking cells a and b. 

Haglund, Haiman and Loehr introduce the statistics Des(a) and Inv(a) to describe the 
nonsymmetric Macdonald polynomials. As in [T], a descent of cr is a pair of entries a{a) > 
a(fe), where the cell a is directly above b. (If b = {i,j), then a = {i,j + 1).) Call Des(a) = 
{a G dg'{'~f) : ^(a) > a{b) is a descent} the descent set and define the leg of u (denoted l{u)) 
to be the number of cells above u in the column of dg{'')) containing u. Let 



maj(a)= ^ {l{u) + l) 



itgDes(CT) 

Consider the cell u = {i,j) and let Cu be the column containing u. The arm of u, denoted 
a{u), is the number of cells to the right of u in row i appearing in columns whose height is 
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Figure 2.1. read{a) =3 23 12456 9, content{a) = {1,2^,32,4,5,6,9} 
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weakly less than the height, h, of Cu plus the number of cells to the left of u in row i — 1 
appearing in columns whose height is strictly less than h. 

Let ai = (ii, ji),a2 = (^25^2)5 and 03 = (is, J3) be three cells in dgi^'j) such that column ii 
is taller than or equal in height to column 22- If ji = J2, ji — is = 1, and ii = i^, then ai, 02, 
and as are said to form a type A triple, as depicted below. 



02 



Define for x, y G Z.). 



I{x,y) 



1 if X > y 
if X < y 



Let a be an augmented filling and let {a(ai), a(a2),S'(as)} be the entries of a in the cells 
{ai, a2, as}, respectively, of a type A triple. The triple {ai, 02, as} is called a tyj>e A inversion 
triple if and only if J(a(ai), a(a2)) + /(a(a2), a(as)) — /(a(ai), a(as)) = 1. 

Similarly, consider three cells {ai = (ii,ji),a2 = (^2,^2), 03 = (^s^Js)} ^ A such that 
column ^2 is strictly taller then column ii. The cells {ai,a2,as} are said to form a type B 
triple if ji = J2, "^2 = "^3, and js — J2 = 1, as shown below. 



ai 



Let a be an augmented filling and let {a(ai), a(a2),S'(as)} be the entries of a in the cells 
{ai, a2, as} of a type B triple. The triple {ai, a2, as} is called a type B inversion triple if and 
only if J(a(as),a(ai)) + /(a(ai), a(a2)) - /(a(as), a(a2)) = 1. 

Let inv(a) be the number of type A inversion triples plus the number of type B inver- 
sion triples. Let coinv(a) be the number of type A 01 B triples which are not inversion 
triples. These two statistics appear in the formula for nonsymmetric Macdonald polyno- 
mials E^ given by Haglund, Haiman, and Loehr [3]. For our purposes it will be more 
convenient to work with the integral form nonsymmetric Macdonald polynomials E^ studied 
by Marshall [6], which are related to the E^ via the equation £'^p_..._^^(x„, . . . , xi; 1/g, 1/t) = 
Ey[Xi, X2, . . . , Xn', q, t). 

Theorem 2.1. [3] The nonsymmetric Macdonald polynomials E^ are given by the formula 



(2.1) 



E,{x;q,t)= Yl ^'^r'^^^^^' 



a maj (ct) jcoinv(o-) 



a: 7— >[7i] 
non-attacking 



n r-, 

^ uedg (7) 

a{u)j^(7{d(u)) 



1-t 



l(u)+l^a{u)+l ' 



where x'' = Ylu&d. 



g'(f,)-^a{u)- 
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We are concerned only with the polynomial E^{x]0,0), so setting q and t equal to zero 
produces the polynomial 

E,(x;0,0) = Yl a;'^0-J(^)0— (^) J] i _ oKl)+tw+i 

a: 7^[n] ■uedg'(7) 

non-attacking a{u)^a{d(u)) 



E 



X" 



a: 7— ►[nj 

non-attacking 

niaj(CT)=coinv(CT)=0 

A non-attacking filling F satisfying maj(F) = coinv(F) = is called a semi-skyline 
augmented filling, SSAF. In the following section, we provide a simpler definition of a semi- 
skyline augmented filling. 

2.2. Semi-skyline augmented fillings. Let F be a semi-skyline augmented filling. The 
condition maj(F) = implies that 

maj(F)= Yl (/(a) + l)=0. 

aGDcs{F) 

Since maj(-F) is increased by at least 1 for each element in the descent set of F, we must 
have Des(-F) = 0. The condition coinv(F) = implies that every triple of cells in F must 
be an inversion triple. Therefore a semi-skyline augmented filling is a non-attacking filling 
with no descents such that every triple is an inversion triple. 

The following two lemmas demonstrate that any filling satisfying these descent and inver- 
sion conditions must be a non-attacking filling. 

Lemma 2.2. Let F be a descentless augmented filling such that every triple of F is an 
inversion triple. If the cells a and b are in the same row of F, then F{a) 7^ F{b). 

Proof. Suppose a and b are in the same row of F. We may assume that a is to the left of 
b. Assume first that the column containing a is weakly taller than the column containing b. 
The cell a is directly on top of some cell c, so {a, b, c} is a type A triple as depicted below. 



The triple must be a Type A inversion triple, so I{F{a),F{b)) + I{F{b),F{c)) — 
/(F(a),F(c)) = 1. We know that F contains no descents, so /(F(a),F(c)) = 0. There- 
fore I{F{a),F{b)) + I{F{b),F{c)) = 1 implies that either F{a) > F{b) and F{b) < F{c) or 
F{a) < F{b) and F{b) > F{c). If F{a) < F{b), then F{b) > F{c) implies that F{a) < F{b), 
for otherwise F{a) = F{b) > F{c), which contradicts the fact that F{a) < F{c). 

Next suppose that the column containing b is strictly taller than the column containing 
a. There must be a cell d on top of b such that {a, d, b} is a type B triple as depicted below. 

"al b 
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There are no descents in F, so F{d) < F{h). Every triple of F is an inversion triple, so 
/(F(d), F(a))+/(F(a), F{b))-I{F{d), F{b)) = 1 implies that I{F{d), F{a))+I{F{a), F{b))- 
= 1. We only need to consider the situation in which F{a) < F{b), which implies that 
J(F(a),F(6)) = 0. This means that I{F{d),F{a)) = 1, so F(a) < F{d). But F{d) < F{b), 
so F{a) < F{d) < F{b) and therefore F{a) < F{b). So F{a) ^ F{b). D 

Lemma 2.3. Let F he a descentless augmented filling such that every triple of F is an 
inversion triple. For each pair of cells a and b in F , with a to the left of b in the row 
immediately below b, we have F{a) ^ F{b). 

Proof. Consider two cells a and b in the augmented filling situated as described. There exists 
a cell d immediately below b and possibly a cell c immediately above a as depicted below. 



c 
a 




b 

d 



If the column containing a is taller than or equal to the column containing b, then a lies 
directly below the cell c which must have F{c) < F{a). Since the triple {c, b, a} is a type A 
triple, it must satisfy /(F(c), F{b)) + /(F(fe), F(a)) - /(F(c), F(a)) = 1. Since F(c) < F{a), 
we have I{F{c),F{a)) = and hence I{F{c),F{b)) + I{F{b),F{a)) = 1. We only need to 
consider the situation in which F{b) < F{a), which implies that I{F{b),F{a)) = 0. In this 
case, I{F{c),F{b)) = 1, so F{b) < F{c) < F{a). This means that F{a) > F{b) and hence 
F{a) ^ F{b). 

If the column containing b is strictly taller than the column containing a, the triple {a, 6, d} 
is a type B triple and must satisfy /(F(6), F{a)) + /(F(a), F{d)) - I{F{b), F{d)) = 1. There 
are no descents in F, so F{b) < F{d), which means that I{F{b),F{d)) = 0. Therefore 
either I{F{b),F{a)) = 1 or J(F(a),F(d)) = 1. If J(F(6),F(a)) = 1, then F{b) > F{a) 
and we are done. Assume I{F{b),F{a)) = and I{F{a),F{d)) = 1. This means that 
F(a) > F{d) > F{b). So F(a) > F{b) and hence F{a) ^ F{b). D 

Corollary 2.4. The descent and inversion conditions used to describe the semi-skyline aug- 
mented fillings are enough to guarantee that the filling is non- attacking. 

Corollary 12.41 follows immediately from Lemmas 12.21 and 12. 3[ It allows us to reformulate 
the combinatorial interpretation of Ery{x; 0, 0) as follows. 

Definition 2.5. Let 7 be a weak composition of n into m parts (where m G Z+ U {c)o}). 
The polynomial E^{x] 0, 0) in the variables x = {xi, X2, ..., Xk) is the formal power series 

E,(x;0,0)= Yl ^''' 

F^SSAFif) 

where SSAF{'j) is the set of all descentless fillings of dg{'-f) in which every triple is an 
inversion triple. 

The combinatorial interpretation of -E(i,o,3,2)(2;; 0, 0) is depicted in Figure [2^ Notice that 
the first row of the i^^ column contains the entry i, since any smaller entry j would attack 
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Figure 2.2. All possible SSAFs of shape (1,0,3,2) with the reading word 
listed below each filling. £'(1^0,3,2) (a^; 0, 0) = xix^xl + xiX2x'^x1 + x\x1x\ + 
a;iX2a;3a;4 + xixlx^x^ + x\x2x1xi + X1X2X3X4 + x\x2Xzx\. 



the basement entry j and any larger entry would create a descent. The following lemma 
characterizes the type B triples in an SSAF. 

Lemma 2.6. // {a, 6, c} is a type B triple in a semi-skyline augmented filling F with a and 
c in the same row and b directly above c, then F{a) < F{c). 

Proof. Let {a, 6, c} be a type B triple situated as described: 



Lemma [2.21 implies that F{a) 7^ F{c). If F{a) > F{c), then consider the cells e and g 
immediately below a and c as shown. 



a 
e 




c 
9 



The descent condition implies that F{a) < F{e), which means that I{F{c),F{e)) = 0, since 
F[c) < F{a) < F{e). Lemma [2.31 therefore implies that F{c) < F{e). Since {e,c,g} form 
a type B inversion triple, we must have /(F(c),F(e)) + I{F{e),F{g)) — I{F{c),F{g)) = 1. 
Therefore + I{F{e),F{g))-I{F{c),F{g)) = 1 implies that I{F{e),F{g)) = 1. By definition 
this means that F{e) > F{g). We now have the same situation as above, but one row lower 
in the diagram. Repeating the argument eventually implies that the basement entry in 
the column containing a is greater than the basement entry in the column containing b, a 
contradiction. Therefore F{a) < F{c). D 
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Given any type B triple {a, b, c} as described in Lemma [2l6l the descent condition implies 
that F{b) < F{c). If F{a) > F{b), then Lemma 12.61 and the fact that the filling is non- 
attacking implies that F (6) < F{a) < F{c). This means that /(F(6),F(a))+ J(F(a),F(c))- 
I{F{b),F{c)) = + — = 0, so {6, a, c} is a type B non-inversion triple. Thus F{a) < 
F{b) < F{c). Therefore Lemma [2.61 completely characterizes the relative values of the cells 
in a type B inversion triple. 

Lemma 2.7. Consider two cells ai = {ii,ji) and 02 = {i2^J2) in on augmented diagram 
dg(7) such that 12 = ii — 1 and J2 < ji- Let 03 = (^3,^3) be the cell directly on top of a2. 
(Then J3 = J2 + I and is = i2-) If F is a semi-skyline augmented filling and F{ai) < F{a2), 
then ^(03) > F{ai). 

Proof. The cells ai, 02, 03 form a type A triple. The filling F must not contain any descents, so 
F{as) < F{a2). If ^(03) < F(ai), then I{F{as), F {a^)) + 1 {F (a,) , F {a2)) - 1 {F {a^) , F{a2)) = 
+ 0-0 = 0^ L However, if ^(03) > F(ai), then 1(^(03), F(ai)) + J(F(ai), ^(02)) - 
/(F(a3), F{a2)) = 1 + - = 1, as required. Therefore ^(03) > F{ai). U 

Lemmas 12. 2[ 12. 3[ 12.61 and 12.71 provide several conditions on the cells in a semi-skyline 
augmented filling. These conditions are used to describe the bijection between semi-standard 
Young tableaux and semi-skyline augmented fillings and to prove Theorem 11.11 

3. Two EQUIVALENT BIJECTIONS BETWEEN SSAFS AND SSYTS 

Recall that we seek a weight-preserving bijection between semi-standard Young tableaux 
of shape A and semi-skyline augmented fillings whose shape rearranges A. Such a bijection 
provides the first combinatorial proof that 

(3.1) 5^E^(x;0,0) = sa(x), 

where the sum is over all infinite weak compositions 7 which rearrange A, denoted 7 ~ A. 
Set Xj = for i > n to obtain the equation 

^ E^(xi, X2, . . . , x„; 0, 0) = sa(xi, X2, . . . , x„). 

7~A 

3.1. A weight-preserving bijection between SSAFs and reverse SSYTs. Assume 
that F is a semi-skyline augmented filling of a weak composition 7 of n into infinitely many 
parts whose largest part is equal to m. Let Ri be the set of entries in the z*'* row of F. The 
collection {-Rj}jg[m] of all sets of row entries is called the row set of F. 

Recall that a reverse semi-standard Young tableau is a filling of a partition shape with 
positive integer values whose entries are strictly decreasing along rows and weakly decreasing 
along columns, where the rows are read from left to right and the columns are read from 
bottom to top. We construct a reverse semi-standard Young tableau p{F) by placing the 
entries from the set Ri into the i*'* row of p{F) in decreasing order from left to right. (See 
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Figure 3.1. The map p{F) 



Figure [3?T1 ) The entries in Ri are distinct since F is a non-attacking filhng. Therefore the 
row entries of p{F) are strictly decreasing. 

To see that the column entries of p{F) are weakly decreasing, notice that any element 
in Ri lies immediately above an element of Ri-i in F. No descents occur in F, so the k^'^ 
largest element, a, in Ri must be less than or equal to at least k of the elements in Ri-i. The 
entry a appears in p{F) immediately on top of the /c*'' largest element, /3, of Ri-i- At least 
k elements in Ri-i are greater than or equal to a, so (3 > a. Therefore the column entries 
of p{F) are weakly decreasing and p{F) is indeed a reverse semi-standard Young tableau. 

The inverse of the map p sends a reverse semi-standard Young tableau P to a semi-skyline 
augmented filling p~^{P) as follows. Assume that the first i rows of P, denoted {Pi, P2, . . . Pi} 
have been mapped to a semi-skyline augmented filling. Consider the largest element, ai, in 
the {i + ly^ row Pj+i. There exists an element greater than or equal to ai in the i*^^ row 
of the SSAF since ai is immediately above such an element in P. Place ai on top of the 
leftmost such element. 

Assume that the largest k — 1 entries in Pj+i have been placed into the SSAF. The /c*'^ 
largest element, ak, of Pj+i is then placed into the SSAF. There are at least k elements of P, 
which are greater than or equal to ak, so at most A; — 1 of these are already directly beneath 
an element of Pj+i. Place ak on top of the leftmost entry j3 in row k — 1 such that jS > ak and 
the cell immediately above /5 is empty. Again, such a (3 exists since the number of entries in 
row Pj_|_i which are greater than or equal to ak is less than or equal to the number of entries 
in row Pj which are greater than or equal to ak- Continue this procedure until all entries in 
Pj-i_i have been mapped into the {i + 1)*^ row and then repeat for the remaining rows of P 
to obtain the semi-skyline augmented filling p~^{P). (See Figure 3.2.) 

We must prove that p~^{P) is a semi-skyline augmented filling, and that this is the only 
SSAF with row entries {Pi}- 

Lemma 3.1. The filling p^^{P) = F' is a semi-skyline augmented filling. 

Proof. By construction, the filling has no descents. Therefore, we must show that all triples 
are inversion triples. First consider a type A triple of cells, {a, 6, c} where a and h are cells 
in the i*'* row and c is the cell immediately below a as depicted below. 
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Figure 3.2. The map p'^{P) 



The column containing a and c hes to the left of the column containing h and therefore must 
be weakly taller than the column containing h. 

The filling is constructed in such a way that F\a) < F'{c). Therefore I{F'{a), F'{c)) = 0. 
We want to show 1 = I{F'{a),F\b)) + I{F'{b),F\c)) - I{F'{a),F'{c)) = I {F' {a) , F'{h)) + 
I{F'{b),F'{c)) - 0. If I{F'{a),F'{b)) = 0, then F'{a) < F'{b), since a and b are in the same 
row of F'. Since F'{b) > F'{a), the entry F'{b) was inserted into F' before the entry F'{a). 
Since F'{b) was not placed on top of c, the entry F'{c) must be less than F'{b). Therefore 
I{F'{b), F'(c)) = 1. So /(F'(a), F'{b)) + I{F'{b), F'{c)) - /(F'(a), F'(c)) = + 1 - = 1 and 
{a, b, c} is a type A inversion triple. 

If I{F'{a),F'{b)) = 1, then F'{a) > F'{b). This implies that F'{b) < F'{c), since F'{a) < 
F'{c). So /(F'(a), F'{b)) + /(F'(6), F'(c)) - /(F'(a), F'(c)) = 1 + 0-0 = 1. So every type 
A triple in F' is a type A inversion triple. 

Next consider a type B triple {a, 6, c} such that a and 6 are in the i*^ row and a is to the 
left of b as shown below. 



Let c be the cell immediately on top of b. Then the column containing b and c is strictly 
taller than the column containing a. We have the equation I{F'[c), F'{a)) + I{F'{a), F'{b)) — 
I{F'{c), F'{b)) = I{F'{c), F'{a)) + I{F'{a), F'{b)) - since F'{c) < F'{b). 

If I{F'{a),F'{b)) = 1, then F'{a) > F'{b). Since F'(c) was not placed on top of F'(a), 
there must be an entry F'{d) greater than -F'(c) which was already on top of F'{a) when 
F'{c) was inserted. If this is the case, then the same situation occurs in row z + 1. Repeat 
this argument until a row r is reached such that the column containing b and c contains an 
entry a in row r but the column containing a does not. (Such a row must exist since the 
triple is a type B triple.) Then the entry a would have been placed on the column containing 
a, which is a contradiction. Therefore I{F'{a), F'{b)) = 0. 

Since I{F'{a),F'{b)) = implies F'{a) < F'{b) and the row entries of P are distinct, we 
have F'[a) < F'(b). Since F'{c) was not placed on top of F'{a), either -F'(c) > F'{a) or there 
was already an entry larger than F'{c) on top of a. If there was already an entry on top of a, 
then the situation is precisely the situation described in the previous paragraph, shifted one 
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row higher. Therefore F'(c) > F'{a) and /(F'(c), F'(a)) + J(F'(a), F'{h)) - I{F'{c), F'{h)) = 
1 + — = 1 and hence {a, b, c} is a type B inversion triple. D 

Lemma 3.2. The SSAF p~^{P) is the only SSAF with row entries {Pi}. 

Proof. Let K be an arbitrary SSAF whose row entries are given by the collection {Pi}. Find 
the lowest row r oi K whose entries appear in a different order from their appearance in 
row r of p~^{P) = F' . Consider the largest element, 5, of Pr whose position in K does not 
agree with its position in F' . Then 5 was placed in the cell on top of the left-most possible 
cell a in F' such that a = F'{a) > 6. Therefore in A' the entry 5 must reside in a cell c of 
a column strictly to the right of the column containing a in K. The situation is depicted 
below, where b is the cell immediately below the cell c containing 6 = K{c) in K and the 
entry 6 is shown as the entry in its cell. 



K F 



row r 
row r — 1 



5 



b 



5 



a 



If the column containing 6 and the cell b is strictly taller than the column containing the 
cell a in K, then the triple {a, b, c} is a type B triple. We have I(S, K{a)) + I{K{a), K{b)) — 
I{S,K{b)) = + I{K{a),K{b)) - 0. We know that K{a) < K{b) by Lemma [Ml So 
I{K{a), K{b)) must equal and hence {a, 6, c} is not a type B inversion triple. 

If the column containing a is weakly taller in K than the column containing b and c, 
then the cell d immediately on top of a in i^ contains an entry K{d) which is less than 
5. (If it were greater than 5, this would contradict the assumption that 5 is the greatest 
value for which the row placement differs.) In particular, K{d) < 5 = K{c) implies that 
I{K{d), K{c)) + I{K{c), K{a)) - I{K{d), K{a)) = + 0-0 = 0. So the triple {d, c, a} is a 
type A triple that is not a type A inversion triple. 

Therefore, regardless of which column is taller, K contains at least one non-inversion 
triple. So K is not a semi-skyline augmented filling. D 

The above implies that the map p~^ sends a reverse semi-standard Young tableau P to 
the only possible semi-skyline augmented filling whose row entries are the same as the row 
entries of P. In fact, let {Ri}j be any collection of sets of positive integers such that the 
number of entries in Ri which are greater than or equal to a is great than or equal to the 
number of entries in -Rj+i which are greater than or equal to a, for alH > 1 and all positive 
integers a. Any set with this property could be the set of row entries for a reverse semi- 
standard Young tableau. So applying the map p produces a semi-skyline augmented filling, 
the only semi-skyline augmented filling with row entries {Ri}j. 

3.2. An analogue of Schensted insertion. Schensted insertion is a procedure for inserting 
a positive integer k into a semi-standard Young tableau T. It is the fundamental operation 
of the Robinson-Schensted-Knuth (RSK) algorithm. We define a procedure for inserting 
a positive integer k into a semi-skyline augmented filling F using a similar "scanning and 
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bumping" technique. In Section H] we use this procedure to describe an analogue of the RSK 
algorithm. 

Let F be a semi-skyline augmented filling of a weak composition 7 of n into m parts. 
Then F = {F{j)), where F{j) is the entry in the j*^ cell in reading order. (Here we include 
the cells in the basement, so j goes from 1 to n + m.) Let j be the cell immediately above 
j. If the cell is empty, set F{j) = 0. We define the operation k ^ F, for k < m. 

Procedure 3.3. The insertion k -^ F: 

PI. Set i := 1, set Xi := k, set Pq = 0, and set j := 1. 

P2. If F{j) < Xi or F{j) > Xi, then increase j by 1 and repeat this step. Otherwise, set 
Xj+i := F{j) and set F{j) := Xj. Set pi = {a,b + 1), where (a, 6) is the j*^ cell in reading 
order. (This means that the entry Xi 'humps' the entry Xj+i from the cell pi.) 

P3. If Xj+i 7^ then increase i by 1, increase j by 1, and repeat step P2. 

P4. Set tfc equal to Pi and terminate the algorithm. 

Notice that this procedure produces a sequence Jfc = (xi, X2, . . . , a;,.) of entries which are af- 
fected by the insertion of /c, called the insertion sequence. The sequence Pk = {pi,P2, ■ ■ ■ ,Pr) 
records which cells are affected by the insertion of k, with t^ (the termination cell) denoting 
the new cell created at the termination of the insertion. This sequence is called the insertion 
path of k. 



Lemma 3.4. Let Xi be an entry in Ik and let Oi be the first occurrence in F of the value Xi 
after pi^i in reading order. Then Xi is inserted into a row above ai by the procedure k ^ F. 



Proof. Let s be the cell containing a^ in F and let a be the cell immediately above s. The 
entry Xi must appear before a in the reading order of F since F is a non-attacking filling. 
(If Xi is Xi, we think of Xj as appearing before all letters of F in reading order.) The entry 
in a must be less than Xj by the choice of Oj. Therefore if Xj reaches a during the insertion 
of k, then x, will replace the entry in a. So x, appears in a row above a^. D 

Lemma 13.41 implies that each entry Xj is inserted into a row of F which is higher than 
the next row containing an entry equal in value to Xj. In particular, this means each entry 
appears in k ^ F above the basement entry of equal value, since every value appears in the 
basement. Therefore the procedure k ^ F terminates in finitely many steps. (See Figure [331 
for an example.) To see that the resulting figure is indeed a semi-skyline augmented filling, 
we need the following lemma. 

Lemma 3.5. The set of entries in a given row of the figure k ^ F is equal to the set of 
entries in the corresponding row of the figure (p(F) ^- k). 

Proof. To prove this, we determine which entry of F is bumped to a lower row of F dur- 
ing (k -^ F) and show that this is the same entry bumped to a lower row of p{F) during 
(p(F)-k). 

If Ri is the collection of row entries appearing in row i of F, let Ri = RiU Xj, where Xj 
is the last entry bumped from a row higher than row i. Let \R~ \ be the number of entries 
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4 = (4, 3) 4 = (4,3,2) 4 = (4, 3, 2) 

Pk = ((4, 3)) Pk = ((4, 3), (5, 2)) Pk = ((4, 3), (5, 2), (2, 1)) 

tk = (2, 1) 

Figure 3.3. The insertion 4 -^ F 



be the number of entries in 



in Ri which are greater than or equal to u. Similarly, let |-Rj^i| 

row i — 1 oi F which are greater than or equal to u. We claim that if v is the largest entry 

in row i such that \R~ \ > l-Rj^'^il, then v is the entry in row i which is bumped down to row 
i — 1. If no such v exists, then no entry is bumped to row i — 1 and the insertion procedure 
is terminated at row i. 

To see this, let v be the largest entry in Ri such that \R~ \ > \R-^i\- If no such v exists, 
then the m*'* largest entry in Ri is less than or equal to m entries in row i — 1, for all m. This 
means that each entry in Ri can be inserted into row i, and so the procedure terminates at 
row i, as predicted by the claim. 

Assume next that such a v exists. Since F is an SSAF, the pigeonhole principle implies 

that |-R~"| < |-Rii"il for E'-ll "W in row i. Therefore the largest v which satisfies \R~ \ > |-Rj^^i| 
must be less than or equal to Xj. If v 



X 



31 



then all the entries greater than or equal to Xj 
in row i — \ are immediately below entries greater than Xj and therefore Xj is bumped down 
to the following row, as claimed. 

If i; < Xj, then there exists an entry in row i — 1 oi F which is greater than or equal to 
Xj but lies immediately beneath an entry, a, less than Xj. During the insertion, Xj bumps 
the leftmost such a. Note that a must be greater than or equal to v since all the entries 
greater than or equal to v in row i — 1 must lie beneath entries greater than or equal to v. 
If a = f , then all the entries greater than or equal to a in row z — 1 lie immediately beneath 
entries greater than or equal to a, so a is bumped to the next row as claimed. If a > v, 
there exists an entry smaller than a that sits on top of an entry greater than or equal to a. 
By the construction of F, this entry is to the right of a, so a will bump this entry. Again, 
since all entries greater than or equal to v in row z — 1 lie below entries greater than or equal 
to t", this entry is greater than or equal to v. The same argument as above shows that either 
this bumped entry is v or this entry bumps another entry in row i. Therefore, the entry v 
is eventually bumped. Since all the entries greater than or equal to v in row i — 1 lie below 
entries greater than or equal to f , the entry v cannot bump anything in row i. Therefore v 
is the entry bumped to the next row down, as claimed. 



14 S. MASON 

Now consider the insertion p{F) ^- k. This insertion procedure scans columns rather than 
rows, but the insertion path moves weakly south as it progresses through the columns from 
left to right. To see this, consider an entry a bumped from row i of p{F). Then a will be 
inserted into the column directly to its right on top of an entry weakly greater than a. The 
entry immediately to the right of a is strictly less than «, so a cannot be inserted on top 
of this entry or at any higher row. Therefore a is inserted into a row weakly lower than the 
row from which it is bumped. 

If a is bumped to a row lower than row i, this means that the set of entries in row i — 1 
which are greater than or equal to a has cardinality less than the cardinality of the set 
containing a and the entries in row i which are greater than or equal to a. Therefore an 

entry a is bumped to a lower row if a is the greatest entry in row i such that \R~ \ > |-R~°i|, 
which is the same condition under which an entry is bumped to a lower row during k ^ F. 
Therefore the set of entries in a given row of the figure k —>■ F is equal to the set of entries 
in the corresponding row of the figure (p(F) ^— k). D 

Proposition 3.1. If F is a semi-skyline augmented filling and k is an arbitrary positive 
integer, then the figure k —* F is a semi-skyline augmented filling. In particular, the insertion 
procedure commutes with the map p in the sense that 

p{k^F) = {p{F)^k), 

where p{F) *— k is the reverse Schensted insertion of k into p{F). 

Proof. Lemma [3.51 states that the row entries of /c — »• F are the same as those of {p{F) <— k). 
The semi-skyline augmented filling p~^{p{F) <— k) is the only semi-skyline augmented filling 
with these row entries by Lemma [3. 2[ so we must prove that p~^{p{F) ^- k) and k ^ F are 
the same semi-skyline augmented filling. In other words, we must show that the row entries 
of A; — >■ F appear in the same positions as in p~^{p{F) ^- k). 

To see this, we prove that every triple in /c — *> F is an inversion triple. Consider first a 
type A triple in A; ^ F, as depicted below. 



a 



c 



Let a,/5, 7 be the entries contained in the cells a,b,c respectively. This triple is a non- 
inversion triple if a < /3 < 7. The cell a cannot be the termination cell for k ^ F. (If it 
were, then c and b would violate Lemma 12.61 in F.) If b were the termination cell, then /3 
must have been bumped from a cell to the right of a for otherwise /3 would have bumped 
a. If the column containing f3 before /3 was bumped were taller than the column containing 
a, then c, the cell containing /3, and the cell below /3 would form a type B non-inversion 
triple in F. Otherwise it must be that a, c, and the cell containing /3 would form a type A 
non-inversion triple in F. Since all triples in F are inversion triples, this is a contradiction. 
Therefore a type A non-inversion triple in /c — > F can only appear if some of the cells in the 
triple were bumped by the insertion. 
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The entries in the cells of the triple are weakly increasing in reading order, so at most one 
of these entries is affected by the insertion procedure. If a bumped a smaller entry, then the 
cells a, 6, and c would form a type A non-inversion triple in F . The entry /3 could not have 
bumped an entry from the cell h for the same reason that it cannot be the termination cell. 
Therefore the only cell in this triple that could be affected by the insertion procedure is c. 
If 7 bumped a smaller entry, k, from the cell c, then k < j3, for otherwise a, 6, c would be a 
type A non-inversion triple in F. 

This implies that the entry k must be smaller than the entry 5 in the cell d directly below 
6, and so the entry e in the cell e directly below c must be smaller than the entry 5, for 
otherwise the cells c, d, e form a type A non-inversion triple in F . But then if 7 appeared 
before /3 in reading order, 7 would have bumped (3. So 7 was bumped from a cell after h in 
reading order. If 7 appeared on the same row of F as 6, the cell containing 7 would form an 
inversion triple with either h and d or d and the cell immediately below 7 in F, depending 
on the column heights. Therefore 7 must have been bumped from a cell on the same row as 
c. Lemma 12.61 implies that the column containing 7 in F must be shorter than the column 
containing a, since 7 > a. The entry above 7 must therefore be greater than (3 so that these 
two cells together with the cell b form a type A inversion triple in F. This means that 7 
must be bumped by an entry greater than jS. Repeating the above argument implies that the 
entry which bumps 7 must come from the same row as 7, and continued repetition implies 
that every entry in the insertion must be bumped from the same row as 7, which contradicts 
the fact that the insertion algorithm begins at the highest cell in the diagram. Therefore 
this situation cannot happen and every type A triple is an inversion triple. 

Next consider a type B triple in /c — >■ F, as depicted below. 



c 



b 



Let Q!,/3, 7 be the entries contained in the cells a,b,c respectively. This triple is a non- 
inversion triple if 7 < « < /5. The cell c cannot be the termination cell for k ^ F. To see 
this, notice that if c were the termination cell, then 7 was bumped from a cell to the right 
of the column containing a. (Otherwise 7 would be placed on top of a.) But if 7 appears to 
the right of a in F, then the cell containing 7, the cell beneath it, and the cell a form a type 
B non-inversion triple in F. 

If the insertion algorithm terminates at the cell a, then the entry a must have been bumped 
from a cell after c in reading order. If a were in the same row of F as c, then Lemma 12.61 
implies that the column containing a is weakly shorter than the column containing c and b, 
since a < (3. But then c, b, and the cell containing a form a type A non-inversion triple. If 
a were in the same row of F as 6, then c, b, and the cell containing a in F would constitute 
a type B non-inverstion triple unless the column containing a were weakly taller in F than 
the column containing b and c. In this case, the entry which bumps a must be less than f] 
since it must be inserted on top of an entry that is greater than (3. (The entry below a in F 
is greater than /3 since these two cells together with b form a type A inversion triple in F.) 



16 S. MASON 

Then we are in the same situation as before and can repeat the argument indefinitely to show 
that no entry is bumped from the row above a, which contradicts the fact that the insertion 
algorithm begins at the first cell in reading order. Therefore a type B non-inversion triple 
in /c — > F can only appear if some of the cells in the triple were bumped by the insertion. 

The entries in the cells of the triple are weakly decreasing in reading order, so at most 
one of these entries is affected by the insertion procedure. If 7 bumped a smaller entry from 
the cell c, then this entry would be less than a and Lemma [2.61 implies that the triple a, b, c 
would be a type B non-inversion triple in F. If a bumps an entry from the cell a, then a 
must be bumped from a cell after c in reading order, for otherwise a would bump 7. If the 
entry a is bumped from the row containing c, then the column containing a in F must be 
weakly shorter than the column containing c and b since a > /3. But then the cell containing 
a together with c and b form a type A non-inversion triple. Therefore a must be bumped 
from the row containing a. Lemma 12.61 implies that the column containing a in F must be 
taller than or equal in height to the column containing c, since 7 < a. The entry above a 
must therefore be greater than 7 so that these two cells together with the cell c form a type A 
inversion triple in F. This means that a must be bumped by an entry greater than or equal 
to 7. Repeating the above argument implies that the entry which bumps a must come from 
the same row as a, and continued repetition implies that every entry in the insertion must 
be bumped from the same row as a, which contradicts the fact that the insertion algorithm 
begins at the highest cell in the diagram. Therefore the entry a could not have bumped 
a smaller entry from the cell a. The entry /3 could not have bumped a smaller entry from 
the cell b, for if so then a, b, c would be a type B non-inversion triple in Fby Lemma 12.61 
Therefore this situation cannot happen and every type B triple is an inversion triple. D 

The insertion procedure k ^ F is closely connected to Schensted insertion. Let reverse 
Schensted insertion be the variation of Schensted insertion which maps a positive integer k 
into a reverse semi-standard Young tableau [8] . This is equivalent to altering the Schensted 
insertion procedure described by Knuth [1] by reversing the directions of the inequalities, 
and scanning columns rather than rows. 

3.3. The bijection ^! between SSYT and SSAF. Let T be a semi-standard Young 
tableau. We may associate to T the word col{T), which consists of the entries from each 
column of T, read top to bottom from columns left to right, as in Figure 13. 4[ In general, 
any word w can be decomposed into its maximal strictly decreasing subwords, called column 
words. This decomposition is called col{w). For example, if 1/7 = 35422 1, then col{w) = 
3 -54 2-2 1. 

Begin with an arbitrary SSYT T and the empty SSAF (p whose basement row contains all 
the letters of Z_|_. Let k be the rightmost letter in col{T). Insert k into (j) to obtain the SSAF 
{k ^ (f)). Then let k' be the next letter in col{T) reading right to left. Obtain the SSAF 
{k' — > (/c — s> 0)). Continue in this manner until all the letters of col(T) have been inserted. 
The resulting diagram is the SSAF \Ef(T). (See Figure [331 ) 

Lemma [3.51 implies that that the entries in the rows of \i/(T) are precisely the entries in 
the rows of the reverse SSYT P obtained by applying reverse Schensted insertion to col(T). 



Figure 3.4. Here col{T) 
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The shape of P is equal to the transpose of the shape of T ([4], [7]). This imphes that the 
shape of \I/(T) is a rearrangement of the shape of T. 

The map "^ is therefore a weight-preserving map from semi-standard Young tableaux of 
shape A to semi-skyline augmented fillings whose shape rearranges A. We now show that \I^ 
is a bijection. 

Note that \1/ is invertible, since reverse Schensted insertion is an invertible procedure and 
\& commutes with reverse Schensted insertion by Proposition 13.11 To describe the inverse 
directly, consider an arbitrary SSAF F. Consider the set S of nonzero columns of F. (In 
Figure [231 S contains the third, fifth, seventh, eighth, tenth, and eleventh columns.) The 
highest cell in each of these columns is a cell which was created during the reverse Schensted 
insertion of the leftmost column of col{T), since the SSAF produced by the insertion of all 
columns except the leftmost column has the shape of the diagram of T with the leftmost 
column omitted. Pick the shortest such column, Ci, where if two columns have equal height 
then the column farther to the right is considered to be shorter. 

The cell Ci at the top of column Ci of F is the cell at which the insertion of the leftmost 
letter of col{T) terminates. Delete F{ci) and scan the reading word in reverse order to 
determine which entry (if any) bumped the entry F{ci). (Any entry, /3, greater than F{ci) 
beneath an entry less than F{ci) would have bumped F{ci).) If such an entry is found, 
replace it with F{ci) and then scan the reading word in reverse order beginning at that 
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Figure 3.5. The map m -.T ^ F takes a SSYT T to an SSAF F. 
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Figure 3.6. The SSAF obtained by removing the 7 in the seventh column of F. 



position to determine the previous bumping entry and replace it with the entry which bumped 
-F(ci). Continue until the first letter of the reading word is reached. The last bumping entry 
is therefore the first entry of col{T). 

For example, in Figure 13. 5^ the entry 7 in the seventh column was the last to be placed 
into F. It was bumped by the 9 in the eleventh column, which was bumped by the 10 in the 
fourth row of the tenth column. This 10 was not bumped from any other position, so the 
10 is the first letter in the word col{T). Removing the 7 in this manner produces the SSAF 
pictured in Figure 13.61 

Repeat this procedure using the resulting SSAF for the second smallest column C2 in S to 
obtain the second letter of col(T). Continue in this manner until the highest cell in each of 
the columns in 5* has been removed. At this point the first column of T has been recovered. 
Then repeat the entire procedure with the new SSAF, whose nonzero columns are each one 
cell shorter than the nonzero columns in the original SSAF. Continue this procedure until 
there are no columns remaining. The resulting word is col{T). 

The map \E' : SSYT -^ SSAF is a weight-preserving, shape-rearranging bijection. This 
proves combinatorially that 

sx{x)= 5^E^(x;0,0), 

7+= A 

since the left-hand side consists of all monomials x'^ where T is a semi-standard Young 
tableau of shape A while the right-hand side consists of all monomials x^ where F is a 
semi-skyline augmented filling of shape 7, where 7"'" = A. 



Let 7 be a weak composition of n into m parts, where m G Z^ U 

The monomials 



X 



3.4. A basis for Q[x] 

{00}. The monomial corresponding to the composition 7 is x''' = Yll 

corresponding to compositions with infinitely many parts form a Q-basis for Q[x], where 

X = {xi,X2,...}. 

Recall that the reverse dominance order on weak compositions 7 and /i is given by 



/^<7 



J2f^i<J2ii 



Wk> 1. 
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Let NK^^ij, be the coefficient of x^ in E^{x]0,0). Then NK^^^ is equal to the number 
of semi-skyhne augmented fiUings of shape 7 and content /x. The bijection presented in 
Section 13.31 imphes that 



',/i' 



7+=A 



where A'a./x is the Kostka number, equal to the number of SSYT of shape A and weight //. 

Proposition 3.2. Suppose that 7 and fi are both weak compositions of n into k parts and 
NK^^^ ^ 0. Then fi < '-/ in the reverse dominance order. Moreover, NK^^^ = 1. 

Proof. Assume that NK^^^ 7^ 0. By definition, this means that there exists a semi-skyline 
augmented ffiling of shape 7 and content /i. Suppose that an entry m appears in one of the 
first m — 1 columns. Then this column contains a descent, since the basement entry of this 
column is less than m. Therefore all entries greater than or equal to m must appear after 
the (m — 1)*^ column. So Yl^iLml^i — Yl^m'^i ^^^ QS-ch. m, as desired. 

Moreover if // = 7, then the entries in the i^^ column must all be equal to i. To see that 
this augmented filling is a semi-skyline augmented ffiling, consider ffist a type A triple as 
shown. 



• 5 



Here 5 > a, so J(a, 5) + /((5, a) — I{a, a) = + 1 — = 1 implies that the triple is indeed a 
type A inversion triple. 

Next consider a type B triple as shown below. 



a . . . 


(3 



Since a < /?, we have I{P,a) + I{a,P) — I{P,P) = 1 + — 0=1. Therefore the triple is 
indeed a type B inversion triple. 

Since all triples are inversion triples and there are no descents, the filling is a semi-skyline 
augmented filling. It is the only SSAF of shape 7 and content 7, so NK^^^ = 1. D 

Corollary 3.6. The polynomials E.y{x; 0, 0) are a Q-basis for Q[x]. 



Proof. Proposition 13.21 is equivalent to the assertion that the transition matrix from the 
polynomials E^{x]0,0) to the monomials (with respect to the reverse dominance order) is 
upper triangular with I's on the main diagonal. Since this matrix is invertible, the E^{x; 0, 0) 
are a basis for all polynomials. D 

Notice that restriction to compositions of n into m parts implies that the E.y{x; 0, 0) form 
a basis for polynomials of degree n in m variables. 
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4. An analogue of the Robinson-Schensted-Knuth Algorithm 

The insertion procedure utilized in the above bijection is an analogue of Schensted inser- 
tion, the fundamental operation of the Robinson-Schensted-Knuth (RSK) Algorithm. 

Theorem 4.1. (Robinson-Schensted-Knuth [7]j There exists a bijection between N-matrices 
of finite support and pairs of semi-standard Young tableaux of the same shape. 

We apply the same procedure to arrive at an analogue of the RSK Algorithm for semi- 
skyline augmented fillings. Recall that Theorem 11.11 states that there exists a bijection be- 
tween N-matrices of finite support and pairs of semi-skyline augmented fillings whose shapes 
are rearrangements of the same partition. Composing the RSK algorithm with the bijec- 
tion $ between semi-standard Young tableaux and semi-skyline augmented fillings proves 
Theorem 11.11 We now provide a direct bijection that commutes with this composition. 

4.1. The map $ : N-matrices — > SSAF x SSAF. Let A = (aij) be an N-matrix with 
finite support. There exists a unique two-line array corresponding to A which is defined by 
the non-zero entries in A. Let aij be the first non-zero entry encountered when scanning 
the entries of A from left to right, top to bottom. Place an i in the top line and a j in the 
bottom line ajj times. When this has been done for each non-zero entry, one obtains the 
following array. 

^ ii 12 ■■ 



wa 

31 32 

Notice that if z,. = v+i, then j^. < j^+i in wa- Every two-line array with this property can 
be obtained from a matrix with finite support. We may therefore consider two-line arrays 
with this property instead of matrices of finite support. 

Procedure 4.2. The map <^:wa — ^ SSAF x SSAF: 

PI. Set r := /, where / is the length of wa- Let F = (p = G, where (j) is the empty SSAF. 

P2. Set F := [jr —>■ F). Let hr be the height of the column in {jr -^ F) at which the 
insertion procedure {jr -^ F) terminates. 

P3. Place V on top of the leftmost column of height h^ — 1 in G such that doing so does 
not create a descent. Set G equal to the resulting figure. 

P4. If r — 1 7^ 0, repeat step P2 for r := r — 1. Else terminate the algorithm. 

Notice that the entries in the top row of the array are weakly increasing when read from 
left to right. This means that if hr > I, placing i^ on top of the leftmost column of height 
hr — 1 ill G does not create a descent. If /i^ = 1, we claim that the z*'^ column of G does not 
contain an entry from a previous step. To see this, argue by contradiction. Assume there 
exists an entry im in the i^'^ column, where m > r. Then the entries jm,jm-i, ■ ■ ■ jr of the 
array wa form a weakly decreasing sequence by the nature of the array. The properties of 
the reverse Schensted insertion of jr imply that the termination of the insertion of jr into F 
must occur in a higher row of F than the termination of the insertion oi jm- This contradicts 
the assumption that ir and i^ are inserted into columns of the same height. 
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Figure 4.1. $ : A -^ ^^AF x ^^AF 



G 



In this way the shape of G becomes a rearrangement of the shape of F . When the process 
is complete, the resuh is a pair (F, G) = ^{A) of filhngs whose shapes are rearrangements of 
the same partition (see Figure |4!TT) . To see that F and G are indeed semi-skyhne augmented 
filhngs, we prove that the map $ produces precisely the pair of SSAFs obtained by applying 
the reverse RSK algorithm to A followed by the map p~^. 

Proposition 4.1. If{P,Q) is the pair of reverse semi-standard Young tableaux obtained by 
applying the reverse RSK algorithm to the matrix A, then {p~^ (P) , p~^ (Q)) = $(A) = (F, G). 

Proof. Proposition 13. II implies that p~^{P) = F. We must prove that p~^{Q) = G, where Q 
is the recording tableau. Consider the insertion of an element ir into G. The position of v 
in G is determined by the termination of insertion (jV — ^ F). Let hr denote the height of the 
column at which the insertion of {jr — ^ F) terminates. The entry ir appears in row hr of G 
by definition. By Proposition 13. H the reverse Schensted insertion of jV terminates in row hr 
of P. Hence v appears in row hr of Q. Therefore the entries in the rows of Q are the same 
as those in the rows of G. 

Recall that p~^{Q) is the unique SSAF whose rows contain precisely the row entries of Q. 
The row entries of Q are inserted into p~^{Q) in decreasing order, so that an entry a appears 
in the leftmost available column such that the entry immediately below a in this column is 
greater than or equal to a. 

The top line of the array wa is a weakly increasing sequence, so its entries are inserted 
into G in weakly decreasing order. Therefore the entries in a given row r are inserted into G 
in decreasing order. An entry v is placed on top of the leftmost possible column which does 
not create a descent. To see that the entries in a given row of p~^{Q) appear in the same 
columns as the entries in a given column of G, argue by induction on the entries in the top 
line of the array wa- 

Consider first the entry ii, where / is the length of Wa- This entry appears in the first row 
of G, in the ij^ column. This entry also appears in the zp column of p~^{Q), since that is 
the leftmost possible position for ii in p~^{Q). 

Assume that the entries ii,ii-i, . . . ,ir appear in the same columns in G as they do in 
p~^(Q). Consider the entry V-i- This entry appears on top of the entry a in p~^{Q). 
Therefore a > ir-i- This fact and the fact that a appears in a lower row of G than ir-i 
imply that a is inserted into G before V-i- Since all the entries which were inserted into 
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G before ir-i appear in the same columns as in p^^{Q), there is no entry on top of a in G 
during the insertion of ir-i- This means that if V-i is not placed on top of a, then V-i is 
placed in a column to the left of the column containing a. Let (3 be the entry under v_i in 
this situation. Then P appears in this same position in p~^{Q). Since V-i is not on top of P 
in p~^{Q), there must be an entry 7 > i^-i on top of P in p~^{Q). But if this were the case, 
then 7 would also be on top of /3 in G since by assumption, all entries greater than V-i are 
in the same positions in G as they are in p~^{Q). This contradicts the assumption that V-i 
is placed on top of p. Therefore the entries of G are in precisely the same positions as in 
p-\Q),sop~\Q) = G. U 

Proposition 14.11 implies in particular that F and G are indeed semi-skyline augmented 
fillings. The semi-skyline augmented filling G records the column to which a cell is added in 
F. Therefore F and G are rearrangements of the same shape. These two facts imply that p is 
a map from N-matrices to pairs of semi-skyline augmented fillings which are rearrangements 
of the same shape. 

4.2. The inverse of the map $. Let (F, G) be a pair of semi-skyline augmented fillings 
whose shapes are rearrangements of the same partition A. Then {p{F),p{G)) is a pair of 
reverse semi-standard Young tableaux of shape A. Since the reverse Robinson-Schensted- 
Knuth algorithm is invertible, this pair {p{F) , p{G)) comes from a unique matrix A of finite 
support. This matrix is $~^((-F, G)). 

We describe the computation of this inverse without mapping through the reverse semi- 
standard Young tableau pair. Let Grs be the highest occurrence of the smallest entry of G 
in reading order. (Here Grs is the element of G in row r and column s.) 

Let s' be the rightmost column of height r in F. Then Frs' is the termination cell for the 
insertion of the last letter, j'l in the array corresponding to the matrix A. Delete Frs' from 
F and Grs from G. Scan right to left, bottom to top (backwards through the reading word) 
starting with the cell directly to the left of Frs' to determine which cell (if any) bumped 
Frs'- If there exists a cell k before Frs' in the reading word such that F{k) > Frs' and 
the cell directly on top of k has value less than or equal to Frs', this F{k) bumped Frs' by 
the argument from Section 1X51 Replace F{k) by Frs' and repeat the procedure with F{k) 
starting from the cell k. Continue working backward through the reading word until there 
are no more letters in the reading word. The resulting entry is the letter ji. (Notice that 
this is the same procedure used in the map \1'~^ in Section 13. 3[ so we have already proved 
that this procedure does in fact yield the inverse of the insertion of ji.) 

Next find the highest occurrence of the smallest entry J2 of the new recording filling. 
Repeat the procedure to find i2- Continue until there are no more entries in the insertion 
and recording tableaux. Then all of the i and j values of the array wa have been determined. 

Corollary 4.3. The RSK algorithm commutes with the above analogue $. That is, if 
{P, Q) is the pair of SSYT produced by the RSK algorithm applied to a matrix A, then 

(VP(P),VP(Q)) = $(A). 
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Proof. The pair (P, Q) maps bijectively to the pair {S, T) of reverse SSYT produced by apply- 
ing the reverse RSK algorithm to A. Proposition 14. II states that {p~^{S), p~^{T)) = [F, G) = 
$(/l). Proposition O implies that ($(P),*(g)) = {p~\S) , p-\T)) , so (*(P),^(g)) = 
(P, G) as desired. D 

The algorithm $ retains many of the properties of the Robinson-Schensted-Knuth Al- 
gorithm. Several of these properties are listed below, although this list is by no means 
complete. 

Corollary 4.4. Assume that <^{A) = {F,G). Then $(A*) = {G,F). 

Corollary 4.5. Let $(A) = (P, G). Let B be a matrix such that the bottom line of wb is 
Knuth equivalent to the bottom line ofwA- Then $(P) = (P, G') for some SSAF G' . 

Corollaries 14.41 and 14.51 follow directly from the fact that the algorithm $ commutes with 
the Robinson-Schensted-Knuth Algorithm. 

4.3. Standardization. A standard Young tableau of shape A is a semi-standard Young 
tableau whose weight is YYi=i ^«' where n = |A|. Any semi-standard Young tableau T can be 
mapped to a standard Young tableau through a procedure called standardization. 

Definition 4.6. Let 7 be a weak composition of n. A skyline augmented filling (SAF) of 
shape 7 is an SSAF of weight HiLi^*- ^^^ -^7 denote the number of skyline augmented 
fillings of shape 7. 

Notice that Py = for certain compositions 7. This means that our analogue of standard- 
ization, skylining, must alter the shape of such compositions. 

Procedure 4.7 {Skylining{F)). Assume that P is an SSAF of shape 7, where I7I = n. 
Consider the reading word as a collection of row words for P. Standardize the reading word 
for P in the usual manner. Then place the new entries back into the original shape of P, 
maintaining the same order. If the lowest entry, aj of the j*^ column is not equal to j, shift 
the entire column to rest immediately on top of the a*'' column. Denote the result by sk{F). 

The entries in the first row of P are always strictly increasing from left to right. There- 
fore the shifting does not permute the order of the non-zero columns. Notice that if a 
appears before c in reading order, then F{a) < F{c) 4^ sk{F{a)) < s/c(P(c)). Therefore 
I{F{a), F{b)) + I{F{b), F{c)) — I{F{a), F{c)) remains the same after skylining. The same is 
true for llFlc),F{a)) + I{F{a),F{b)) - I{F{c),F{b)) since F{a) < F{c). This means that 
skylining preserves the inversion triples. This fact and the fact that there are no descents 
within the columns of P imply that the figure sk{F) is indeed a semi-skyline augmented 
filling. 

Proposition 4.2. The bijection \i/ commutes with standardization. 

Proof. Let T be a semi-standard Young tableau and \E^(T) the corresponding SSAF. We saw 
in section 13.11 that the SSAFs are in bijective correspondence with reverse SSYTs, so it is 
enough to consider the reverse SSYT R corresponding to T. 
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Figure 4.2. An SSAF F and its standardization sk(F) 



The set of entries in a given row r oi R are sent to a new set of row entries under 
standardization. However, the new entries are the same entries which appear in row r of the 
skyhning of \1/(T) by the definition of s/c(\l/(T)). There is only one SSAF with the same row 
entries as the standardization of R, so this must be s/c(\l/(T)). D 
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